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ABSTRACT'. In the age of technology, educators are encouraged to incorporate online resources 
into their teaching, but the effectiveness of these resources on learning and the student perspective 
is rarely taken into consideration. A key aspect to the assessment of online resources for 
international students is the user’s perspective. Culture has a significant impact on preferences 
towards tools, design aesthetics, and information provided online. This study begins with a review 
of theories and research on how students learn across oral, written and online modes of 
communication, creating a foundation for assessing online resources. After highlighting the 
similarities and differences across modalities, the study looks at current assessment methods as 
noted in scholarly literature. The study concludes by exploring student perceptions through a case 
study of three adult international students. With this information, educators are able to select and 
assess effective online resources for diverse groups. This is a very broad overview of theories and 
research. It is not intended to go into much depth on any particular aspect, but rather to encourage 
educators to reflect on their use of online resources. 
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Conceptualizing the Assessment of Online Resources 

The advent of the internet quickly and dramatically expanded knowledge and its social 
nature, giving people the ability to take advantage of intellectual capital (Leu, Kinzer, 
Coiro & Cammack, 2004). Today there is an app for almost anything and online 
resources have made their way into the classroom. However, the assessment of these 
resources is often overlooked. A common practice in education is to consider a student’s 
prior knowledge when teaching a concept; likewise, it is equally important to consider the 
needs of diverse students when selecting and assessing online resources. Without 
acknowledging the student perspective, educators miss a key aspect of their students’ 
intellect, causing students to struggle to engage and maneuver through online 
information. In contrast, meaningfully connecting with information online promotes 
problem-solving and critical thinking, helping students become confident and responsible 
contributors of information (Detlor, Booker, Serenko, & Julien , 2012). In reviewing the 
evolution of communication across modalities, educators can build a foundation for 
understanding how to use and assess online resources effectively. As society has 
developed from oral traditions, to writing, to online media, the modes of communication 
have diversified how students engage in social learning and have broken down barriers, 
but what has not changed is what it takes to learn (Laurillard, 2002). Although various 
theories are presented in this study, educators are encouraged to reflect on their personal 
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beliefs of how students learn across modalities and the needs of diverse groups to connect 
those concepts to assessment methods. 

Life Before the Alphabet: Orality 

Through oral traditions or orality, society first began to share information. Centered on 
events or interactions, the meaning of words in orality derives from the interplay between 
what is being said and its context (Goody & Watt, 1968). The interpretation of 
experiences is mostly in the moment, since revisiting information is highly dependent on 
one’s ability to recall what was said. Learning stems from immersion in the words of 
others, repeating and mastering the oral discourse of a field in action (Ong, 1982). 

People use discourse to develop and present ideas persuasively and in a favorable style of 
speaking— while incorporating ideas of others and seeking connections between ideas. By 
actively participating in storytelling, people act out and identify with societal values and 
beliefs (Levi-Strauss, 1966). Information changes as new experiences challenge the 
assumptions used to interpret the original experience. Reflection is essential because it 
helps differentiate and integrate the context of interpretations to create greater meaning 
(Mezirow, 1991). Oral information changes through the process of forgetting, 
transmitting extraneous information, conveying information inconsistently or glossing 
over information from person to person (Goody & Watt, 1968). In orality society passes 
down information from one generation to another through memorized stories or accounts 
of events. 

Immortalizing Words in Writing 

The development of an alphabetic script was a technological advancement, which 
decreased the dependence on memory to transmit and retain information (Levi-Strauss, 
1966). Although the written word loses the sensory impact of a person-to-person 
emotional connection as it shifts to a person-to-written representation of a person’s 
words, writing enhanced orality by providing a means to organize concepts (Ong, 1982). 
Since writing is a slower process than speaking, it enables the written word to be less 
fragmented (Chafe, 1982). People are able to explore ideas and events through 
classifying, sequential ordering, and explaining with greater ease (Ong, 1982). This 
process leads to corrections and resolutions of inconsistencies. In capturing spoken 
words, writing is a key factor in enabling society to transmit thought and culture over 
time and place (Good, 1977). The evaluation of texts, in addition to oral discourse, 
brings light to varying ideas, values and beliefs. 

Going Online 

As the development of writing was a technological advancement on orality, online 
technology advanced both the spoken and written word. The meaning of online 
information still derives from the interplay between what is being said [online] and its 
context. Knowledge is experienced with the spoken and written word through the use of 
a computer and it is still socialized. Speaking, reading, writing, and virtual play is used 
to act out and identify with societal values and beliefs. Again, there is a loss of 
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interpersonal connection as communication shifts to person-to-digitized representations. 
The internet creates an environment conducive to the exchange of life experiences and 
different points of views (Coiro, 2009). The exchange, like in all modalities, transforms 
old ideas and builds new ones through contemplation and reflection. Online repositories 
store information throughout time, place and now space as knowledge is taken out of the 
physical world and placed online, further extending its shelf life and arguably its 
accessibility. Purposefully engaging in problem solving online requires one to know 
what to pay attention to and what to ignore, especially with features that attempt to 
capture one’s attention (Lankshear & Knobel, 2001). The complex online environment 
requires regulator strategies to skillfully transition between rapid reading, searching, and 
the in-depth construction of meaning (Coiro & Dobler, 2007). Online resources and 
applications help record, store, and share information. 

Assessment of Online Resources 

Interaction with others is essential in all modalities for critical thinking, connecting 
knowledge to practice, problem solving, and innovative thinking. Without this 
foundation, the acquisition of information becomes, “a fonn of consumption without the 
production of deep knowledge and the development of skills important for the future” 
(Gee, 2012, p. 419). The modes of communication are conducive to learning when they 
present content in a meaningful way, encourage students to think and create, give 
students the freedom to make choices on how to interact in the environment, and are 
collaborative and social (Gee, 2012). As modalities are embedded in society, exercising 
these modalities differs across cultures. Meaning, the way in which one speaks, writes, 
or functions online, differs across countries, generations, status, etc. This difference in 
meaning is especially important to remember when selecting and assessing online 
resources for diverse groups. 

In beginning any assessment process, it is important to remember the purpose of 
assessment is to improve student outcomes. What it means to have an understanding of a 
particular subject differs across subjects, departments, institutions, countries, etc. 
Assessment practices should be situated in culturally and institutionally supported 
description of what it means to have an understanding of the particular subject. A review 
of assessment literature in two relating fields, online courses and electronic platforms, 
identified (a) relevance to students’ needs, (b) connection to academic concepts, (c) 
freedom and self-directedness, (d) organization, (e) collaboration, (f) accuracy and 
creditability, and (g) cost effectiveness, as key areas of assessment. Incorporating 
principles of assessment into educational practice can lead to the meaningful selection of 
resources, improved use, and enhanced learning. 
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Student Needs and Perceptions 

The online world adds another dimension to a student’s environment and 
communications with others. In the online environment, students learn from interacting 
with and using programed instructional systems, which require the learner’s needs to be 
anticipated and addressed in advance (Govindasamy, 2001). This makes monitoring 
students’ interaction and use of programed instructional systems essential in the 
assessment of online resources (Gonzalez & Westbrock, 2010). To gain a better 
understanding about how learners are using online resources, Welch (2007) suggests 
reviewing Webserver logs to measure reference-generated visits. After tracking the use of 
their online resource, Courtois, Higgins, and Kapur (2005) looked at student perceptions 
and success by constructing a single-question survey that asked if students found the 
resource helpful. Laurillard (2009) emphasizes the importance of considering why the 
learner should participate and creating a series of activities that keep learners engaged. To 
discover the needs of learners Grays, Del Bosque, and Costello (2008) used virtual focus 
groups to assess the value of their online resource. Student needs and perceptions can be 
assessed through web logs, single-question surveys and focus groups. 

Academic Concepts 

Online resources are most effective in connection with academic concepts, with clearly 
stated objectives and features that guide learners through the lesson (Dewald, 1999). 

Since the online environment is the mode in which students are engaging in learning, it is 
important to incorporate learning theory within the programed instructional system. A 
study by Rebb and Gibbons (2004) stipulate that if online resources were more 
experiential, learners would find them more beneficial, understand the context of the 
information, and connect with broader concepts. The methods of assessing learning 
theory and bridging to academic concepts online are as varied as assessing teaching in 
traditional classrooms. Machine evaluations, such as multiple-choice test and simulations 
can be effective assessment methods (Ehlers, 2013). Ehlers (2013) states that comments 
posted by students and product-based assessments can be used to examine student work 
created with the aid of an online resource. It is also useful to review the technology 
affordances, which McCracken, Cho, Sharif, Wilson, and Miller (2012) define as, 
“mapping technology to the kinds of interactions that lead to learning” (p. 108). To 
examine the academic concepts presented in online resources one can review technology 
affordances, multiple-choice tests, simulations, comments, and product-based 
assessments. 

Self-Directedness and Prior Knowledge 

The amount of information available online requires learners to cultivate self- 
directedness. Resources should guide students through the learning process by helping 
them establish connections between repositories of knowledge and academic concepts. 
Laurillard (1996) stipulates that the access rates of online resources should be tracked and 
the information used to motivate learners. Sun and Rueda (2012) investigated the impact 
of computer self-efficacy and self-regulation in student engagement in distance education 
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by surveying students using a questionnaire adapted from the Motivated Strategies for 
Learning Questionnaire; Situational Interest Scale; and Web Users Self-Efficacy Scale. 
The results indicated interest and self-regulation positively correlated with all types of 
engagement. With sufficient self-directedness and motivation, learners can often 
compensate for a lack of prior knowledge. Coiro and Dobler (2007) argue that with 
access to information, individuals with high levels of online reading skills may 
compensate for low levels of prior knowledge. The varying levels of prior knowledge 
and skills of learners should be taken into consideration when using online resources. 
Lessons should begin with self-assessments to see if students have already learned the 
concepts being presented; self-assessments also help students recognize key concepts and 
the material’s organization (Ehlers, 2013). The online resources’ capacity to cultivate 
self-directedness and prior knowledge can be judged using access rates, surveys and self- 
assessments, like quizzes and check-off lists. Self-assessments come in many forms and 
include brief quizzes and short check-off lists. 

Organization 

Since an abundance of information is stored on multiple platforms, it can be difficult for 
students to navigate resources and differentiate from academically irrelevant resources. 
Educators can fall prey to presenting an overwhelming amount of information or 
presenting a moderate amount of information in a disorienting way. Ehlers (2013) 
suggests resources that are simple, use intuitive navigation and aid in processing 
information. The organization should be regularly reviewed for relevance, ease of use, 
and fonnat consistency. A learner’s time should be focused on critical reflection, not 
spent lost in resources. Jackson and Pellack (2004) developed a self-assessment survey 
for institutions to assess their online resources. Hosie, Schibeci and Backhaus (2005) 
created a checklist to assess the quality of learning materials, by reviewing: accessibility, 
currency, richness, purpose, and inclusivity. Bowles-Terry, Hensley, and Hinchliffe 
(2010) developed best practices for creating video tutorials, by looking at speaking pace, 
video length, content, aesthetic, findability, and student interest. The organization of 
online resources can be evaluated through surveys and checklists. 

Collaboration 

Novice and experts alike interact with, contribute and alter information online. These 
interactions and contributions are critical to helping learners understand concepts through 
connectivity with others. Online resources can be enhanced by using various online 
collaboration applications to give learners the opportunity to engage the resource’s 
author, other novices and experts in discourse on concepts. Through discourse, 
information and academic concepts are transformed and learners learn. Ehlers (2013) 
believes that good online resources offer multiple opportunities to connect students with 
their teacher, peers, and other experts. This fosters a community of practice and 
communicative learning principles, while challenging learners to investigate the accuracy 
and credibility of sources. The design of online resources should be conducive to 
collaboration in a natural and authentic way. Collaboration can be gauged through 
monitoring comments, emails, or other student contributions. 
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Accuracy, Credibility and Cost 

Since both information and the online environment changes regularly, it is necessary to 
ensure online resources are accurate and creditable. Anyone can place information online 
without formal review, making it essential to examine the claims and assumptions 
presented in resources. Morain and S warts (2012) developed several rubrics to access 
various aspects of video tutorials to include the video’s accuracy, completeness, and 
pertinence; and the author’s confidence, self-efficacy, and engagement. Hosie, Schibeci 
and Bachaus (2005) created a checklist to review the reliability of the interface, learning 
goals, directions, communication, bandwidth, accessibility, and style of the resource. As 
technologies are ever changing, it is important to analyze the costs of resources to the 
potential impact on student outcomes. Dobbs and Sittler (2013) evaluated the economic 
value of online resources with a rubric that looked at the cost per use, goals, usage, visits; 
and subscription traffic. A resources’ accuracy and cost-benefit can be calculated using 
rubrics and checklists. 

Implementation 

With so many methods of assessing various elements of online resources, it may be 
difficult to design an appropriate approach. The time dedicated to assessment should be 
proportionate to the intended use. A two-minute video should not be assessed to the 
same extent as a costly district- wide resource. First, it must be reiterated that the purpose 
for assessment is to improve student outcomes. Next, it is important to consider a 
culturally and institutionally supported description of assessment elements. Then, 
educators should briefly check through the elements that the resource addresses. This 
article highlights relevance to students’ needs, connection to academic concepts, design 
suited for self-directedness, organization, collaboration, accuracy, and cost effectiveness. 
An online resource does not need to cover all of these elements to be considered a “good 
resource.” Unless the teaching is taking place completely online, these resources are 
imbedded in instruction. Any element that is missing can be incorporated into the overall 
lesson. However, if the online resource is not substantively contributing to the intended 
task, its use should be reconsidered. Lastly, depending on the resource’s use, complexity, 
and cost, educators should look at each or a selection of the assessment elements in 
greater depth, using a coordinating assessment method. It is essential that educators take 
the time to assess online resources, making implicit considerations explicit. 
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International Student Perceptions of Online Resources 

Online behavior is a cultural expression of a community’s shared values (Chau, Cole, 
Massey, Montoya-Weiss, & O'Keefe, 2002). For international students, this means that 
their authentic online behavior may be significantly different from the perspective 
presented in a US classroom. The dominant culture of the class can disrupt student 
learning by cutting students off from a valuable source of information- their culture. To 
maintain student-centered practices, educators need to keep the student perspective in 
mind (Rogers, 1969); this matter most closely relates the assessment element, student 
needs and perceptions. When multicultural! sm is promoted in education, diversity 
becomes a resource not a disadvantage (Waters, 2001). 

Conceptual Framework 

With little research published on student perceptions of online resources, literature on 
customer perceptions of websites in the international business was reviewed. Chau, Cole, 
Massey, Montoya-Weiss, and O'Keefe (2002) defined culture as a collective phenomenon 
of the shared values of a community in their study on consumer online behaviors. Jin 
(2010) contends that culture is a determining factor in the websites people find attractive. 
Seidenspinner and Theuner (2007) believe that culture determines preferences towards 
navigational tools, design aesthetics, and information provided. Mazaheri, Richard, and 
Laroche (201 1) connect emotions to perceptions. Emotions lead to the perception of the 
degree to which the site is considered informative, effective and entertaining. This 
transition from emotions to perception then influences one’s attitudes and involvement. 

In the conceptual framework of this study, culture (values and beliefs) shapes perceptions 
(how one interprets experience), which then informs preferences. The following are 
some useful definitions for the framework: 

• Perceptions: an interpretation of the experience 

• Culture: values and beliefs that distinguishes one group of people from another 
(including- gender, race, socio-economic status) 

• Values: characteristics that are identified as important 

• Beliefs: general assumptions or ideas held to be true 

• Preferences: a demonstration of like or dislike for one characteristic over another 

Research Questions 

(1) How do international students use online resources for academic purposes? 

(2) What process do international students use to select online resources? 

(3) What qualities do international students value? 

(4) What types of resources do international students prefer? 

These questions address the study’s goals by providing insights needed to guide the 
selection of meaningful online resources and provide a basis of assessment. 
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Data Collection 


This study used interviews and a document analysis in a five-part data collection process. 
The students: 

(1) were given an overview of the research project and signed informed consent, 

(2) had up to thirty minutes to explore the mock research question online while their 
actions were recorded, 

(3) were interviewed for fifteen minutes about the steps they took to answer the 
research question, 

(4) responded to some general questions about their online activity, and 

(5) were thanked for their participation and asked to be available in the future for 
follow up questions. 

The mock research question, “Is wind energy cheap? Is it effective? Is it practical?” was 
constructed by considering the students’ language level and vocabulary. The question 
was out of the students’ area of expertise to encourage the need to research the answer. 

Participant Selection and Site 

Students were selected to participate in this study based on their country of origin and 
language proficiency. Since the study explores international student perceptions of 
online resources, it was imperative to select participants from diverse backgrounds. 
However, without interpreters and translated materials, the students had to have advanced 
language proficiency to communicate with the researcher and actively participate in the 
study. For this reason, all three participants were selected from a list of former students 
of a small language program associated with a liberal arts college. Student identities 
were kept confidential and pseudonyms were used. Han is a twenty-year-old, Korean 
student. He received his high school diploma in his home country before coming to the 
US a year ago. Mo is twenty-seven years old, Saudi Arabian student. He received a 
bachelor’s degree in his home country before coming to the US a year ago. Charlie, a 
nineteen-year-old Taiwanese student, received his high school diploma before coming to 
the US nine months ago. 

The location of the data collection, a small conference room at a local coffee shop, was 
selected based on the accessibility and familiarity to the participants. The conference 
room was formal enough to provide a structured setting, but casual enough to not impede 
responses. The interviews were not conducted in a classroom, since the halls of the 
school are very busy and the presence of their peers might have affected their responses. 
The location also provided Wi-Fi access and power outlets, which was essential for the 
study. 
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Data Analysis 

After completing the fieldwork, the data were analyzed with the aid of Atlas. ti. The 
analysis began with a review of the students’ answers to the mock research question. 

Next, a document analysis was conducted on each student’s recorded online activities. 

The document analysis was compared to the students’ interview responses for agreement, 
using both a deductive and inductive coding approach. Various codes were assigned to 
highlight distinct commonalities and differences amongst the students and between the 
interviews and the online recordings. Responses were coded that appeared to be 
culturally significant and provided insights into the formation of student perceptions. 

This form of data analysis was selected because it allows for a synthesis of multiple 
responses and captures various points of view. 

Results 

During the interviews, the students indicated they move between English and native 
language resources when using the internet for academic purpose. However, each student 
transitions between languages a bit differently. Charlie says that he searches in English, 
“because the question comes from English not from my language”. Then he switches to 
Mandarin, “if I cannot find the things I want, I cannot read or I don’t want to read”. Han 
says that he prefers starting with Korean resources. He reasons, “Korean is my native 
language, so I can understand easier than English, but sometimes it’s the same, even if I 
cannot understand Korean I can use English”. Han uses English resources as a backup, 
when Korean sites fall short of providing him with the information he needs. Overall, a 
transition in language is initiated when the student hits a wall and feels another language 
can provide additional information. 

Although all of the students mentioned in their interviews that they use native language 
resources while conducting research, none of the students used them in their recorded 
search for this study. This finding was interesting and could have many possible causes. 
When Charlie was asked why he does not use a translator during his university science 
class, he said that it would not be fair since the other [native English speaking] students 
could not use translators. He went on by saying, “I finish my English program, I should 
know things and I don’t. . . I feel like I’m the only one, it’s only me.” For Charlie, there 
seems to be a certain level of shame associated with openly using native language 
resources for academic purposes and he may not want to stand out amongst his peers as 
different. 

The process that the students use to select online resources centers mostly on information 
provided by others rather than internal criteria. All three students used search terms that 
came directly from the mock research question. Mo reasoned, “Because it was the first 
title, first part of the question”. Charlie and Mo used Google’s auto complete by typing 
in the first word on the mock research question then accepting Google’s suggestion to 
complete the search phrase. Once the information query was complete, Charlie and Mo 
selected the first site listed by Google. Han, on the other hand, read the site descriptions 
to make his selection, and chose the fifth resource. He stated, “I was looking for like a 
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sentence about my thesis statement, some sort of I thing like a sentence and also what is 
the reason in the research paper and then what information I want to take from this 
paper”. Charlie and Mo seem to trust the information presented by Google with little 
critical scrutiny, while Han is more selective and considers the relationship between the 
search term and the purpose of the research. 

Popularity seems to be a quality that the students value. All three students used Google 
in their recorded search and stated it was their preference. Mo mentioned that he uses 
Google because it is the most popular site in the US and easy to use. Charlie said he 
prefers Google, but also uses Yahoo to search in Mandarin. He deemed Yahoo a better 
search engine because “it is more popular in Taiwan and yielded better results”. Lastly, 
Han said he uses www.naver.com, a popular search engine in South Korea. The students 
expressed a connection between popularity and the quality of information (if many 
people use it, it must be good). 

The students had varying ideas about the characteristics of resources that they liked or 
disliked. “If there is too much information at the same time,” Han finds the resource 
repelling. Charlie does not like resources that make it difficult to find information or 
read. He determines the quality of a resource by comments made by other people, “if 
somebody writes a comment right and maybe like the comment shows some of the same 
problems the article have or like this article has some part is like error”. He goes on to 
say he dislikes resources that are political, while Mo dislikes advertisements. Overall, the 
students seem to like or dislike a resource by how the information is presented (quantity 
and organization) and the content (advertisements and politics). 

Limitations 

During the study the following threats to validity were considered: 

• Is there another basis for the student’s perceptions that has not been identified? 

• Have the student’s perceptions been misidentified? 

• Can the student’s clearly articulate their perceptions? 

• Do the interview questions target the student’s perceptions? 

• Does the researcher’s bias lead to a misinterpretation of the student’s perceptions? 

• Does the researcher’s presence cause the student’s to alter their responses? 

These questions are threats to validity because they highlight possible misinterpretation 
of the data, drawing incorrect conclusions, or not being able to draw conclusions at all. 

In order to guard against these threats the study was designed to use various activities to 
investigate the same questions. This confirms the information collected and highlights 
relationships. During the study, the students were asked to share any additional 
information that came to their minds, allowing students to consider and include aspects 
not explicitly asked in the study. Each response was rephrased and repeated back to the 
student to provide an opportunity to confirm the interpretation of the information 
provided. The use of computer-recorded actions allowed students to review their steps 
and reflect on their actions after completing them. These computer-recorded actions 
lessen the student’s dependency on memory and were used as a basis of comparison. An 
outside advisor monitored this process and provided continual feedback. 
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Although the results of this study can inform the selection of online resources for the 
participants, the information should be generalized cautiously. While generalization can 
be made about various cultures, culture is unique to each individual. The values and 
beliefs of one person from a particular cultural group may not be representative of the 
entire group. The participants in this study have been in the US for nine months to a 
year, which could have impacted the cultural practices of the students. Furthermore, 
culture and the online environment are dynamic and change over time. The results of this 
study are emblematic of the moment in time in which the study took place and caution 
should be taken when generalizing the results to future dynamics. 


Conclusions 

The international students in this study indicated that they move between native language 
and English resources. Being able to transition between languages to find additional 
information can be a significant advantage for international students, adding richness to 
the student’s research. Trusting Google, selecting the first site listed, and favoring 
popular resources is not unique to international students. According to an American 
Library Association report (1989), students across the nation need to learn the skills 
needed to locate, evaluate and effectively use information resources. However, 
international students may be more sensitive to the quantity and organization of 
information in resources. In order for diverse students to engage in the critical 
exploration of academic knowledge, educators need to consider whose narrative or 
version of the truth is being taught and how the student will relate to this information 
(Phan & Baurain, 2011). It is important to understand each student’s educational history 
and create assignments that give students the ability to draw on their prior knowledge to 
reduce information overload (Miller & Endo, 2004). To create a nurturing and conducive 
learning environment, educators need to consider the assumptions they make regarding 
student online behaviors and honor the student’s culture and preferences. 
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